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1.0  INTRODUCTION 


This  is  a  draft  test  plan  developed  to  apply  ^^®j^'®p‘4s'’arroperated 
necessary  to  evalnate  pilot  lenlity.  tL 

rniririft^ef ‘rSpl^l^lSd  ?Lrs^rnL°irsr-luarioAethods, 

data  processing  techniqnes,  and  eLi^^ent .  A 

concerning  pilot  performance  __-.ns  connected  with 

second  letter  report  detailed  the  xssues  ^fomance  and 

using  visual  flight  simulation  for  Lvironment. 

developing  target  levels  of  safety  state-of-the-art  helicopter 

That  report  emphasized  assessing-  existi  ^  .  availability,  and 

simulators  with  regard  to  capabilities,  Hraft  test  plan  provides 

basic  user  cost.  The  addition  of  a  ^^^^^^trLen^s  Lcessary 

the  FAA  with  a  tool  to  evaluate  the  simulat  ^  levels  of  safety, 

to  assess  pilot  performance,  perception,  and  targe 

This  draft  test  plan  outlines  the  and  objectives 

reoorts  to  provide  a  clear  understanding  of  the  scope  a  J 

needed  to  perform  the  simulation.  Using  an  qLerated  to 

model,"  both  real  and  abstract  heliport  ^®o^gtacle-rich 

test  a  pilot;  s  ability  “  “aSmfSm  be  eLluated,  including 

heliport  environment.  Each  flight  ^  .  fo-around  requirements, 

scenarios  that  address  ground-to-air  and  f  J^^tor^nd 

Resultant  data  will  be  used  to  assess  per  soecified  flight 

associated  levels  of  safety  within  each  of  for 

reaimes.  Final  simulation  results  will  be  providea 
“Cification  through  actual  aircraft  flight  testing. 

The  overall  project  is  divided  into 

Simulation  requirements  and  assesses  eritLia  to 

focuses  on  the  application  and  ana  foiio-wina  separate  subtasks 

develop  the  actual  simulation  test.  The  following  separate 

form  the  core  of  phase  one: 

1.  test  and  evaluation  requirements/ 

2.  simulation  requirements  and  facilities, 

3.  simulation  test  plan,  and 

4.  pilot  briefing  materials. 

Thorough  exainination  of  the  results  of  subtasks  1  P|°f  p®*" 

thC  foundation  for  test  plan  develo^ent  To  supplement  the  test 
plan,  subtask  4  provides  pilot  briefing  materials  that  expia 
pilot's  role  and  participation  in  the  test  program. 

1.1  PURPOSE 

This  test  plan  is  designed 

evaluate  pilot  performance  using  ^  .  Usina  the  simulation 

CaSbfeCTefihfdlrrCS^ 

fhf C“Cer°ft,"vrrHSpCC?' CbJCacLCprcrEhvironment  Simula 


Requirements  and  Facilities,"  a  model  plan  will  be  formulated  for 
visual  scenario  development  and  optimal  simulator  effectiven 


1.2  STRUCTURE 

To  sustain  a  smooth  transition  between  test  specifications  and 
simulation  requirements,  each  criterion  identified  ''^^hin  the  tesr _ 
plan  must  be  carefully  weighed  and  matched  against  current  ^ 
technology.  Comprehensive  simulation  modeling  that  will  dup  ic 
tlsk  scenarios  within  acceptable  levels  o£  fidelity  will  be  carefully 
examined.  Development  of  each  facet  of  the  test  plan  must 
appropriately  address  the  necessary  tier  of  simulation  complexity. 

To  provide  the  FAA  with  an  assessment  of  the  most  cost-effective 
simulation  facilities,  the  information  gathered  from  the  '  IFR  Visual 
Segment  Evaluation  Test  Plan"  was  used  as  the  nucleus  for  this 
plan.  Numerous  reviews  and  on-site  evaluations  were  conducte  ^ 

the  developmental  stage  of  the  IFR  test  plan  to  identify  suitaPie^ 
simulation  facilities.  Both  projects'  simulator  specificatior^.  The 
Obstacle-Rich  Environment  Assessment,"  and  the  "IFR  Visual  Segment 
Evaluation"  have  similar  test  requirements.  Both  have  a  visual  sc 
condition  as  the  common  core  which  establishes  the  basic  Simula  ion 
requirements: 


o  fidelity  of  object  recognition,  _ 

o  orientation  capability  and  ability  to  align  for  arrival  or 

departure,  and 

o  vehicle  flight  handling  qualities . 


1.3  BACKGROUND 

The  evolution  of  simulation  technology  offers  engineers  a  tool  with 
qualities  that  will  closely  duplicate  actual  aircraft  in  any  flight 
regime.  The  advancement  of  rotorcraft  simulation  has  accelerate  ^ 
the  past  decade.  Most  simulation  facilities  have  unique  engineering 
capabilities  with  specialized  hardware  and  software  that  maintain 
control  of  evaluation  test  parameters.  They  provide  a  complete 
service,  including  the  system  and  skilled  personnel  to  modify  or 
develop  the  simulation  to  satisfy  test  requirements. 

2.0  OBJECTIVES  AND  ISSUES 


2.1  OBJECTIVES 

The  objective  of  the  proposed  test  plan  is  to  develop  a  method  for 
qualifying  and  quantifying  pilot  performance  in  an  obstacle-rich 
environment.  Using  human  factors  engineering  techniques,  the  true 
man/machine  interface  (MMI)  can  be  explored.  Execution  of  designatea 
pilotage  tasks  will  provide  a  weighted  workload  versus  overload 
assessment  of  performance.  The  test  results  will  assist 
planners.  Development  and  enhancement  programs  at  urban  heliports 
airports,  influenced  by  nearby  obstacles,  will  be  able  to  control  an 
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manage  the  impact  of  those  obstacles  on  their 

distinction  between  current  and  future  aerodrome  weiahted 

will  be  the  application  of  these  results.  By  ^PP^^^JJ  takeoff  and 
factor  to  visual  obstacle  clearance  criteria  wit  i 
landing  area  of  a  heliport  or  airport,  minimal  operational 
interference  can  be  accomplished.  This  methodology  provi 
solution  to  dense  obstacle-rich  environments. 

The  ability  of  the  helicopter/pilot  combination  to 
visual  conditions  is  well  understood  and  helicopter  per  ... 

various  takeoff  and  landing  modes  is  well  define  .  ®  ouantified. 

the  heliport  under  varying  obstacle  density  has  ^^vSuation 

This  established  the  need  for  conducting  the  simulator  evaluation 

described  in  this  test  plan. 

The  effect  that  obstacles  have  on  a  helicopter  pilot's 

maneuver  the  helicopter  may  impact  heliport  design  and  operations  in 

several  ways. 

o  Applying  visual  approach  clear  zone  dimensions 

obstruction  clearance  requirements  can  strongly  influence  t 

treatment  of  obstacles  and  the  protected  and 

a  heliport.  In  addition,  recognition  of  well-defined  arri 
departure  corridors  within  the  heliport  VFR  imaginary  surface 
could  strongly  influence  minimum  acceptable  levels  of  obstacle 

density. 

o  The  visual  cues  within  the  obstacle-rich  Lf  ® 

actual  landing  or  departure  may  prove  to  be  insuffic  ff  of 

pilot  to  execute  the  procedure.  This  may  influence  t  ^esu^t 

close-in  obstacles  on  protected  airspace  requirements  and  result 

in  unacceptable  levels  of  safety. 

o  The  ability  to  see  and  avoid  obstacles  while  evaluating  their 
effect  on  pilot  performance,  perception,  and  ® 

seen.  Effects  on  minimum  protected  airspace  requireme 

illustrated. 

The  absence  of  well-defined  simulation  procedures 
measuring  human  factors  initiatives  provides  an 

to  explore  innovative  techniques  for  assessing  pilot  performs  . ' 
pLSinary“ndications  support  that  cutrent 

revising  protected  heliport  airspace  requires  new  ^  .  1 

new  procedural  assessments  will  bring  into  focus  a 

and  operational  issues  dealing  with  pilot  could 

resolved.  Applying  test  results  to  ”in-use  VFR  urban  helipo 

have  the  consequence  whereby  pilot  prof iciency  or  ^  Current 

standards  may  be  required  to  operate  at  a  oS^aSe- 

takeoff  and  landing  paths  in  and  out  of  VFR  facilities  in 
rich  environment  could  exceed  these  performance  / 

rendering  a  heliport  an  operational  hazard.  Applying  the 
performance  standards  to  in-use  facilities  would  increase  the 
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difficulty  when  considering  certification  requirements.  Clarification 
will  reouire  a  site-by-site  assessment  to  find  an  appropria  e 
resolution  for  each  facility.  A  secondary  objective  of  the 
evaluation  is  application  to  in-use  facilities.  Using  real  helipo 
models  as  part  of  the  simulation  study  will  generate  a  preliminary 
baseline  to  apply  these  measures  to  "actual"  heliports. 

The  major  objective  of  this  test  plan  is  to  determine  what  effect  an ■ 
obstacle-rich  environment  has  on  pilot  performance,  percep  ion, 
safety.  The  helicopter  pilot's  ability  to  maneuver  the  aircraft 
obstacle-rich  environment  significantly  affects  cockpit  wor  oa  . 
These  simulator  tests  are  designed  to  determine  the  limits  to  w  i 
helicopter  pilots  may  be  expected  to: 


o  become  (and  remain)  oriented  under  varying  atmospheric  and  visual 
conditions  within  the  heliport  environment  or  transversing  the 
arrival  or  departure  corridors. 


o  reliably  see  and  avoid  obstacles  underneath  the  visual 

approach/ departure  corridor  leading  up  to  and  away  from  t  e 
heliport,  and 

o  take  advantage  of  human  factors  techniques  to  determine  effects  of 
obstacle  density. 


2.2  ISSUES  TO  BE  EVALUATED 


The  issues  associated  with  this  undertaking  are  in  five  basic, 
interrelated  simulation  areas:  visibility,  airspace  dimensions, 
helicopter  maneuverability,  psychological  effects,  and  safety, 
are  specific  problem  areas  associated  with  each  of  these  issues. 


There 


2.2.1  Visibility 


These  issues  relate  to  the  pilot's  ability  to  become  and  remain 
visually  oriented  while  transitioning  from  an  arrival  or  depar  ure 

pattern . 


2. 2. 1.1  Object  Visibility 

How  readily  visible  to  the  pilot  are  the  various  categories  of  terrain 
features  and  obstacles  in  an  urban  environment?  Are  there  distinct 
and  measurable  differences  in  the  ability  to  perceive  . 

features,  buildings,  bridges,  towers,  poles,  wires,  helipad  marki  g 
and  lights,  and  various  lighted  and  unlighted  structures. 


2. 2. 1.2  Approach  Speed 

How  does  helicopter  approach  speed  affect  the  pilot ' s  ability  to 
discern  obstructions  and/or  landmarks  in  sufficient  time  o  a 
appropriate  action  at  various  visibility  levels? 
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2. 2. 1.3  Orientation/Situational  Awareness 

How  much  familiarity,  obstruction/landmark  recognition,  area 
definition,  etc.,  is  required  for  the  pilot  to  be  certain  he/she  will 
be  able  to  remain  within  the  protected  VFR  imaginary  surfaces  provi 
for  the  approach  or  departure? 

2. 2. 1.4  Familiarity 


What  effect  does  familiarity  have  on  reduced  visibility  limits  for  the 
approach / departure ? 

2.2.2  Airspace  Dimensional 


These  issues  relate  various  factors  to  the  protected  airspace 
dimensions  that  must  be  allocated  to  assure  a  safe  procedure  for  aii 
authorized  users. 


2. 2. 2.1  Object  Visibility 

What  are  the  impacts  of  variations  in  object  visibility  and 
discernability  on  the  minimum  required  dimensions  of  the  visual 
orientation  zone  (at  varying  visibility  levels)? 

2. 2. 2. 2  Familiarization  Aids 

How  do  factors  such -as  improved  charting,  area  familiarization, 
distinctively  painted/lighted  structures  and  beacons,  and  helipad 
marking/lighting  affect  protected  airspace  dimensions? 


2. 2. 2. 3  Maneuvering  Speed/ Air space 

Do  reductions  to  maneuvering  speeds,  while  navigating  within  the 
visual  segment,  affect  protected  airspace  dimensions? 

2. 2. 2. 4  Transition  Airspace 

How  much  area  must  be  protected  to  allow  lateral  transition  through 
the  heliport  environment  during  arrival  or  departure? 

2.2.3  Helicopter  Maneuverability 

These  issues  are  directly  related  to  the  pilot^  s  ability  to  maneuver 
the  helicopter  as  intended. 

2 . 2 . 3 . 1  Course  Acquisition 

How  quickly  will  the  pilot  be  able  to  capture  the  intended  visual 
course  (e.g.,  heliport  environment)  once  it  is  visually  identified. 
What  aids  will  help? 
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2. 2. 3. 2  Maneuvering  Speed 


To  what  extent  does  reduced  speed  enhance  maneuverability  and  at  what 
speed,  if  any,  does  control  become  a  problem? 

2. 2. 3. 3  Maneuverability 

How  quickly  or  safely  can  the  pilot  maneuver  to  avoid  obstacles  within 
the  visual  approach /departure  corridor? 

2.2.4  Psychological  Effect  on  Pilot  Response  and  Performance 

The  test  plan  will  answer  some  basic  pilot  performance  questions  about 
the  number,  type,  and  placement  of  obstacles  found  below  the  VFR 
heliport  imaginary  surfaces .  Does  obstacle  density  create  a 
psychological  consequence  that  outweighs  the  desirability  of  a 
heliport  location? 

2. 2. 4.1  Risk  Assessment  for  Pilot  Decisions 

Flight  near  heliports  presents  a  certain  level  of  risk  in  striking 
obstacles.  Can  a  potential  risk  assessment  factor  be  assigned  to  a 
pilot's  decision  that  would  yield  a  numerical  weighted  value  for 
analysis? 

2. 2. 4. 2  Obstacle  Perception  Factor 

Each  pilot  mentally  allocates  a  level  of  intimidation  to  the  potential 
hazard  that  an  obstacle  may  present.  Consequently, • there  is  a 
supplemental  factor  that  must  be  considered  when  rating  obstacles  ot 
different  texture  and/or  consistency  and  their  potential  effect  on  a 
pilot's  perception. 

2.2.5  Target  Level  of  Safety 

Safety  introduces  the  need  for  analytical  quantification  of  obstacles 
affecting  pilot  performance  to  generate  a  fundamental  "target  ° 

safety  (TLOS)."  A  TLOS  provides  an  objective  way  to  measure  and 
manage  safety. 

A  level  of  safety  needs  to  be  established  regarding  the  number  and 
proximity  of  obstacles  near  a  VFR  heliport  so  that  potential 
psychological  effects  on  pilot  performance  can  be  minimized. 

2 . 2 . 5 . 1  Risk  Factor  for  Obstacle  Density 

No  specific  evaluation  process  has  been  designed  to  test  obstacle  _ 
density  under  the  VFR  imaginary  surfaces  from  a  perceptual  basis,  can 
the  perception  of  the  potential  risk  of  individual  obstacles  be 
measured  by  type,  location,  height,  and  intensity  to  yield  a  numerica 

value? 
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2 . 2 . 5 . 2  Risk  Factor  For  Close-In  Obstacle  Perception 

The  appearance  of  close-in  obstacles,  as  determined  by  their  relative 
distance  and  position  in  the  immediate  landing  and  takeoff  area, 
provides  a  true  perspective  of  the  overall  obstacle  scene. 

Limitations  of  visual  cues  within  the  surrounding  f ield-of-view  may 
adversely  affect  a  pilot's  ability  to  assess  an  appropriate  level  of 
avoidance.  The  design  and  development  of  the  various  scenarios  for 
the  test  must  represent  the  potential  significance  of  close-in 
obstacles. 

2.3  SIMULATION  OBJECTIVES  AND  SCOPE- 

To  arrive  at  a  productive  test  syllabus,  the  specific  objectives  of 
the  simulation  effort  must  be  defined. 

2.3.1  Visibility  and  Object  Recognition 

The  objectives  are  to  ascertain  the  pilot's  ability  to  visually 
discriminate  and  recognize  various  classes  of  objects  and  obstacles 
under  varying  visibility  conditions.  This  includes  evaluation  of 
various  types  of  objects  (terrain,  buildings,  towers,  bridges,  etc.), 
the  heliport  and  its  environment,  and  lighting  aids  and  other  visual 
cues  under  measured  reduced  visibility  conditions. 

2.3.2  Orientation  and  Alignment 

Given  appropriately  designed/defined  approach  and  departure 
environments,  the  ability  of  the  pilot  to  become  quickly  oriented  rn 
the  operational  environment  must  be  evaluated.  This  is  critical  where 
considerable  alignment  maneuvering  may  be  required  because  the 
heliport  or  departure  corridor  does  not  lie  straight  ahead.  This 
objective  includes  evaluation  of  all  candidate  techniques  for 
orientation  enhancement,  including  specialized  charting  improvements, 
specialized  landing  aids,  and/or  familiarization  training. 

2.3.3  Maneuvering  and  Control 

This  objective  includes  quantifying  pilot  and  aircraft  performance  to 
provide  the  heliport  developer  with  an  explanation  of  the  methodology 
used  for  determining  the  airspace  to  be  protected  for  rotorcraft  when 
maneuvering  to  the  visual  approach/departure  corridor  or  transitioning 
to  the  heliport.  These  determinations  must  consider  the  entire  range 
of  possible  positions  both  inside  and  outside  the  95  percent 
probability  corridor.  Also  included  is  the  need  to  evaluate  the 
minim\mn  maneuvering  speeds  for  which  the  helicopter  is  certificated 
and  at  which  the  pilot  feels  comfortable  in  visual  corridors. 

2.3.4  Pilot  Performance  and  Perception 

The  basic  assumption  is  that  pilot  performance  and  perception  or 
"workload"  is  affected  by  a  variation  in  the  density  of  obstacles .  It 
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is  evident  that  an  "overload"  will  occur  at  some  point  based  on  task 
complexity  and  difficulty.  The  objective  is  to  induce  conditions  of 
overload  through  simulation  that  hampers  performance  capability. 

Rating  scales  based  on  the  1969  Cooper  and  Harper  evaluations  will 
provide  an  appraisal  of  the  simulator  events. 

Currently,  the  scope  of  the  simulation  study  is  limited  to  the 
collection  of  data  only,  with  analysis  of  that  data  to  be  accomplished 
in  later  tasks .  The  intent  is  to  collect  sufficient  data  to  provide 
statistically  significant  results  in  each  of  the  four  above-mentioned 
areas  so  they  may  be  used  as  the  basis  for  developing  appropriate 
weighted  measures  in  an  obstacle-rrich  environment . 

3.0  APPROACH  PROCEDURE  MODEL 

Two  types  of  approach  procedure  models  are  to  be  evaluated  during 
these  sdLmulation  studies: 

o  real  model,  in  which  actual  urban  heliport  environments  are  re¬ 
created  with  actual  approach/departure  corridors,  facilities,  and 
charting;  and 

o  abstract  model,  in  which  a  variety  of  variables  may  be  selected, 
either  as  modifications  to  a  real  environment  or  to  an  environment 
created  specifically  for  the  evaluation,  or  both. 

The  re-creation  of  real  environments  will  provide  an  important  link  to 
actual  operational  experience  for  validation  of  the  results,  as  well 
as  a  test  bed  for  evaluating  alternatives  pertinent  to  those ^ 
environments.  The  abstract  model  will  be  important  for  providing  the 
ability  to  evaluate  issues  of  interest  specific  to  this  study. 
Complexity  of  the  evaluation  and  the  number  of  variables  must  be 
explored.  A  motif  queue  of  three  core  prototype  patterns  will  be 
needed  to  create  as  many  as  30  scenarios.  These  scenarios  will 
include  visibility,  airspace  dimensions,  obstacle  avoidance 
capability/techniques  (maneuverability) ,  psychological  effects  and 
safety.  The  processing  flow  chart  for  each  model  is  shown  in 
figure  1. 

3 . 1  REAL  MODEL 

The  proposed  real  model  is  based  on  two  urban  heliports  that  are  in 
common  use  today.  The  first  set  is  to  be  based  on  the  Indianapolis 
Downtown  Heliport  that  is  located  on  irregular,  uneven  terrain  in  an 
area  with  only  a  few  significant  obstructions  to  flight  or  visibility. 
A  second  set  would  involve  the  Wall  Street  Heliport  that  is  located  on 
a  specially  built  pier  in  the  East  River  near  numerous  tall  buildings, 
a  series  of  bridges,  and  constant  harbor  vessel  traffic.  While  the 
Wall  Street  location  is  far  more  problematic  operationally,  the  local 
pilot  population  is  accustomed  to  operating  in  that  environment . 
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FIGURE  1  MODEL  PROCESSING  FLOWCHART 


Each  of  these  locations  presents  a  different  set  of  issues  to  be 
addressed.  In  Indianapolis,  where  the  best  approach  direction  is  from 
the  southwest  (because  of  obstacles),  the  approach/departure  corridors 
interfere  with  departure  traffic  from  the  city's  nearby  commercial 
airport.  An  easterly  approach  or  departure  satisfies  the  air  traffic 
control  problem.  Unfortunately,  it  involves  negotiating  close-in 
obstacles  which  reduce  heliport  operability . 

The  Wall  Street  Heliport  also  presents  a  variety  of  additional 
problems  associated  with  helicopter  capability  and  pilot  performance: 
slow  speed  requirements,  minimum  turn  radius  for  arrival  and 
departure,  steep  descent  and  climb  gradients,  and  a  highly  obstructed 
environment . 

3.2  ABSTRACT  MODEL 

The  abstract  model,  which  may  be  based  on  an  actual  or  generic 
heliport  environment,  is  intended  to  provide  the  variety  of 
operational  variables  needed  to  evaluate  the  relevant  issues.  To 
accomplish  this,  the  following  features  must  be  generated  and  be  at 
the  command  of  the  simulator  operator: 

o  a  complete  urban  model  with  buildings,  highways,  rivers,  natural 
terrain  features,  and  major  landmarks  such  as  sports  stadiums, 
etc . ; 

o  a  heliport  and  landing  environment  appropriate  to  the  urban 

environment,  created  with  minimum  size,  marking,  lighting,  etc.; 

o  current  and  experimental  charting; 

o  a  set  of  candidate  heliport  and  visual  lighting  options; 

o  a  pool  of  selectable  obstacles  (including  buildings,  towers,  and 
terrain  features)  designed  to  control  or  infringe  on  the  VFR 
heliport  imaginary  surfaces  to  varying  degrees,  from  anywhere  in 
the  arrival  or  departure  corridor; 

o  a  pool  of  landmarks  suitable  for  evaluating  pilot  ability  to 
identify  the  pre-landing  zone  environment,  and/or  to  serve  in 
providing  visual  guidance  to  the  landing  zone;  and 

o  a  standardized  set  of  reduced  visibility  levels  to  be  created  by 
the  simulator  visual  subsystem. 

4.0  SIMULATOR  REQUIREMENTS 

4.1  SIMULATION  FEASIBILITY 

State-of-the-art  simulation  technology  has  resulted  in  a  variety  of 
applications  for  both  rotary-  and  fixed-wing  aircraft .  The  main 
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consideration  is  achieving  the  desired  fidelity  for  the  specific 
simulation  initiative  being  evaluated.  Achieving  simulation  that 
emulates  the  required  variables  is  paramount.  Any  simulation  must 
qualify  and  quantify  all  necessary  aspects  of  the  flight  test  regime. 
The  characteristics  described  in  the  following  paragraphs  are  the 
minimum  conditions  a  simulation  candidate  must  emulate  to  verify  a 
procedure . 

4.2  SIMULATION  CHARACTERISTICS 

4.2.1  Rotorcraft  Handling  Qualities  and  Dynamics 

The  difficulty  in  designing  helicopter  simulation  models  increases 
proportionately  with  the  complexity  of  components  and  modules 
represented. 

4. 2. 1.1  Dynamics 

A  single-engine  helicopter  model  that  accurately  portrays  actual 
aircraft  characteristics  over  the  full  range  of  flight  regimes  must  be 
used  in  this  simulator  investigation.  The  degree  of  sophistication 
required  of  these  models  clearly  depends  upon  the  flight  performance 
characteristics  to  be  simulated.  Looking  only  at  the  VFR  flight 
segment  allows  specific  simulation  parameters  to  be  refined. 

4. 2. 1.2  Handling  Qualities 

To  stay  within  the  parameters  of  this  task,  the  qualities  addressed  in 
the  following  paragraphs  must  be  established  in  a  simulation 
rotorcraft  model. 

Responsiveness  -  The  simulator  must  respond  properly  to  changes  in 
attitude,  altitude,  temperature,  gross  weight,  center  of  gravity, 
configuration,  and  ground  effect. 

Control  Forces  -  The  control  forces  and  degree  of  travel  must 
accurately  represent  the  helicopter  being  modeled.  If  a  generic 
helicopter  is  modeled,  the  control  forces  must  represent  similar 
helicopters  in  the  same  performance  class . 

Response  to  External  Influence  -  The  simulator  must  be  capable  of 
providing  representative  modeling  of  crosswind,  wind  shear,  and 
air  wake  effects  caused  by  terrain,  buildings,  etc. 

Response  Rates  -  The  simulator  must  demonstrate  acceptable  and 
consistent  response  rates  as  reflected  in  the  cockpit 
instrumentation,  visual,  and  motion  systems  (if  provided) . 

4.2.2  Visual  Aids 

In  the  heliport  environment,  visual  aids  provide  course  accuracy  by 
supplying  cues  to  the  pilot  for  estimating  his/her  position  in 
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relation  to  the  heliport.  Visual  systems  are  not  perfect  and  do  not 
provide  true  position  relative  to  the  earth.  For  simulation,  visual 
aids  must  represent  the  operating  and  error  characteristics  of  the 
particular  system  used  when  executing  arrival  or  departure  procedures 
from  a  heliport . 

4.2.3  Cockpit  Instrumentation  ' 

4 . 2 . 3 . 1  Instrument  Capability 

The  simulator  must  provide  sufficient  instrxmentation  to  support  the 
execution  of  both  arrival  and  departure  procedures. 

4 . 2 . 3 . 2  Navigation  and  Communication  Equipment 

The  simulator  must  have  navigation  and  communication  capabilities  that 
would  support  normal  VFR  operations  and  require  normal  cockpit 
interface.  The  navigation  equipment  must  have  operational  and 
accuracy  characteristics  representative  of  the  functions  and 
performance  capabilities  of  simulated  systems . 

4 . 2 . 3 . 3  Performance  Instrumentation 

The  simulator  must  possess  performance  instriimentation  (engine, 
transmission,  torque,  etc.)  that  would  require  normal  cockpit 
attention. 

4. 2. 3. 4  Instrument  Response 

Instrument  response  must  correlate  to  the  rate  of  change  and 
displacement  of  controls  of  the  helicopter  being  modeled. 
generic  helicopter  is  being  modeled,  the  response  must  be  typical  of 
similar  single-engine  helicopters  in  the  same  performance  class. 
Instrument  response  must  be  Closely  coupled  to  the  visual  system. 

4.2.4  Environmental  Disturbances 

The  simulator  must  be  capable  of  emulating  the  ceiling  and  visibility 
conditions  associated  with  a  visual  approach  or  departure.  This  must 
include  varying  weather  phenomena  such  as  changes  in  wind  direction 
and  velocity,  precipitation,  haze,  and  smoke. 

4.2.5  Visual  Scene  Response 

The  visual  reproduction  must  emulate  various  conditions  necessary  to 
validate  the  test  initiatives .  The  parameters  described  in  the 
following  paragraphs  are  a  minimum. 

4 . 2 . 5 . 1  Vi'sual  Scene  Quality 

Visual  scene  quality  and  the  ability  to  accurately  depict  reduced 
visibility  are  the  foremost  considerations  for  this  evaluation. 
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4. 2. 5. 2  Visual  Scene  Response  Time  (Transport  Lag) 

The  visual  scene  must  respond  to  abrupt  pitch,  roll,  and  yaw  at  the 
pilot's  position  within  100  milliseconds  of  the  time  when  the 
helicopter  would  respond  under  the  same  conditions. 

4 .2 . 5 .3  Resolution 

The  simulator  must  be  capable  of  depicting  a  variety  of  obstructions 
with  sufficient  resolution  (3  arc  minutes  or  better  at  the  pilot's 
eye)  to  support  detection,  identification,  and  avoidance  capability  in 
accurately  presented,  reduced  visibility  models. 

4. 2. 5. 4  Field  of  View 

The  nature  of  this  evaluation  requires  a  minimum  field  of  view  of  90 
degrees  (150  degrees  preferred)  horizontal  and  +,  20  degrees  (i  40 
preferred)  vertical. 

4. 2. 5. 5  Day,  Night,  Dusk 

The  simulation  will  require  evaluation  during  day,  night,  and  dusk 
operations.  Buildings  and  other  obstacles  must  be  capable  of  being 
properly  (realistically)  lighted  for  dusk  and  night  operations. 

4. 2. 5. 6  Depth  Perception 

The  simulator  must  provide  necessary  visual  cues  to  allow  assessment 
of  sink  rates  and  sufficient  depth  perception  for  low  altitude/ low 
airspeed  maneuvering,  hover,  and  landing. 

4.2.6  Motion  and  Sound  Cues 

Motion  is  desirable  but  not  required  for  this  evaluation.  If  motion 
is  available,  it  must  conform  to  actual  aircraft  response  to  external 
and  internal  inputs .  It  should  exhibit  characteristic  buffet  where 
applicable.  The  simulator  must  produce  sounds  corresponding  ii^ 
amplitude  and  frequency  to  sounds  found  in  the  represented  cockpit. 

4.2.7  Cockpit  Viewing  Angles 

The  pilot's  viewing  angles  must  represent  the  helicopter  being 
modeled.  If  a  generic  helicopter  is  used,  the  viewing  angles  should 
represent  helicopters  in  the  same  class.  Unobtrusive  visual  shields 
representing  cockpit  structural  members,  glare  shields,  etc.,  may  be 
used  to  block  the  pilot's  vision  where  appropriate. 

5.0  SIMULATION  DATA  COLLECTION  METHODOLOGY 

The  following  paragraphs  describe  the  generic  test  conditions  and  the 
data  elements  to  be  recorded  during  the  tests .  These  test  conditions 
and  data  elements  to  be  recorded  will  be  developed  in  greater  detail 
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during  phase  two  of  the  test  progrcuti  when  specific  knowledge  of  the 
simulation  facility  is  known. 

5.1  SIMULATOR  VARIABLES  AND  TEST  MATRIX 

This  section  recommends  simulator  variables  that  will  be  used  in 
developing  the  simulation  test  plan.  Specific  scenario  design  should  . 
concentrate  on  the  following  elements: 

o  •  visibility, 
o  terrain  types, 
o  type  heliport /helipad, 

o  lighting  options, 

o  airspeed, 
o  course  orientation, 
o  obstacle/ structures,  and 
o  meteorological  conditions. 

Precise  scenario  modeling  requires  further  conditional  statements  when 
considering  the  abundance  of  obstacles  and  flight  regimes.  Designing 
the  actual  simulation  demands  a  clear-cut  organization  of  obstacles 
and  the  flight  regimes  into  distinctive  fields.  The  following  are  to 
be  considered. 

5.1.1  Obstacle  Classes 

When  considering  the  number  and  type  of  obstacles  associated  with  the 
simulation  development  the  following  category  definitions  will  be 
applied: 

o  obstacles  (below  the  VFR  surface)  -  landmarks  and  visual  cues  used 
for  orientation,  navigation  and  flight  progress  monitoring, 

o  obstacles  (below  the  VFR  surface)  -  perceived  as  potential 
threats,  and 

o  obstacles  (above  the  VFR  surface)  -  to  be  avoided. 

Mote:  -  Obstacles  will  not  conform  to  or  define  the  8:1  VFR  slope. 

5.1.2  Flight  Phases 

The  arrival  and  departure  corridors  can  be  considered  a  thoroughfare 
in  the  sky.  To  provide  assurance  that  the  appropriate  aspect  of 
flight  is  evaluated  against  a  specific  class  of  obstacle  the 
succeeding  classifications  will  utilized: 

o  Orientation  Phase 

o  visual  (en  route)  to  visual  approach  transition, 
o  acquisition  of  approach  course  based  on  visual  cues,  and 
o  obstacle  avoidance  in  the  transition  areas. 


14 


o 


Arrival  Phase 

o  approach  course  already  visually  acquired, 

o  cues  use  for  navigation,  speed  and  descent  rate  control,  and 
o  obstacle  avoidance  along  approach  course. 

o  Landing/Takeoff  Phase 

o  landing  zone  already  visually  acquired, 
o  cues  used  for  final  descent  and  landing  control, 
o  cues  used  for  takeoff  and  departure,  and 
o  obstacle  avoidance  in  immediate  heliport  environment. 

5.1.3  Issues  in  Relation  to  Obstacle  Classes 


The  preliminary  assessment  of  issues  must  be  matched  against  the  above 
obstacle  classes  and  flight  phases.  This  link  provides  the  groundwork 
for  scenario  design  and  development  when  matched  against  distinct 
simulation  variables. 


o  obstacles /landmarks /cues 

o  orientation  phase 

o  arrival  phase 

o  landing/takeoff 

o  obstacles  perceived  as  threats 
o  orientation  phase 
o  arrival  phase 
o  landing/takeoff 
o  obstacles  to  be  avoided 
o  orientation  phase 

o  arrival  phase 


o  landing/takeoff 


associated  issues 

2. 2. 1.1, 
2. 2. 1.4, 
2. 2. 4. 2. 

2. 2. 1.2, 
2.2.3. 1, 

2. 2.1. 3, 
2. 2. 3. 2. 

2. 2. 1.1, 
2. 2. 1.4, 

2. 2. 1.2, 
2. 2. 3.2, 

2. 2. 1.3, 
2. 2. 4. 2. 

2. 2. 1.1, 
2. 2. 1.4, 

2. 2. 1.2, 
2. 2. 4. 2, 

2. 2. 1.3, 
2. 2. 5. 2. 

associated  issues 

N/A 

2. 2. 4.1, 

2. 2. 4. 2, 

2 . 2 . 5 . 1 . 

2. 2. 5. 2, 

2. 2. 4.1, 

2. 2. 4. 2. 

associated  issues 

2. 2. 1.1, 
2. 2. 1.4, 
2. 2. 3. 2, 

2. 2. 1.2, 
2. 2. 3. 3, 
2. 2. 4. 2. 

2. 2. 1.3, 
2. 2. 3.1, 

2. 2. 1.1, 
2.2 .1.4, 
2. 2. 2. 3, 
2. 2. 3. 2, 

2.2. 1.2, 
2. 2. 2.1, 
2. 2. 2. 4, 
2. 2. 4. 2. 

2’.2.1.3, 
2. 2. 2. 2, 
2. 2. 3. 3, 

2.2 .1.1, 
2.2 .1.4, 
2. 2. 4. 2, 

2. 2. 1.2, 
2. 2. 3. 3, 
2. 2. 5. 2. 

2. 2. 1.3, 
2. 2. 3. 2, 
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By  categorizing  test  objectives  with  scenario  variables,  issues  of 
perception,  performance,  and  safety  can  be  assessed.  In  the  test 
plan,  development  scenarios  and  estimates  of  the  number  of  simulator 
test  runs  required  will  be  evaluated  against  all  of  the  scenario 
variables.  Two  tables  are  presented  for  planning  purposes.  Table  1 
categorizes  the  test  objectives  (a  through  j)  and  lists  the  variables 
that  are  to  be  evaluated  in  order  to  address  each  of  the  issues 
identified  in  section  2.2.  Table  2  presents  estimates  of  the  number 
of  simulator  tests  runs  that  will  be  required  to  evaluate  all 
simulation  variables . 

The  key  to  the  number  of  runs  and  scenario  development  is  the  use  of 
three  prototype  patterns .  Each  pattern  is  designed  to  provide  a 
baseline  on  which  to  focus.  This  will  maintain  scenario  integrity  and 
control  throughout  test  runs  as  variables  are  entered  or  removed  from 
the  individual  scenes.  Each  will  establish  a  foundation  from  which 
the  real  or  abstract  model  can  be  generated.  It  is  anticipated  that 
somewhere  within  the  range  of  a  prototype  pattern  will  exist  a  real 
heliport  model.  The  abstract  heliport  models  will  be  composed  of 
addition  (or  deletion)  and  modification  of  features  and  obstacles 
pertinent  to  the  specific  test  objectives  (a  through  j) . 

Each  of  the  test  cases,  a  through  j,  will  be  examined  to  verify 
prototype  pattern  and  model  suitability.  Using  a  real  model  will 
validate  pilot  performance,  perception,  and  safety  in  a  verifiable 
heliport  environment .  Real  models  are  limited  in  scope  and  may  not 
fully  present  the  appropriate  test  scenario.  Abstract  models  furnish 
that  experimental  influence  necessary  to  probe  variations  in  obstacle 
density. 

Subject  pilots  will  be  teamed  in  small  groups  of  three.  Each  pilot 
will  perform  the  same  number  of  runs  on  a  given  test.  Tests  will  be 
sequenced  randomly  to  prevent  undue  subject  pilot  familiarity  with  the 
simulated  environment  (except  those  for  which  familiarity  is  being 
specifically  evaluated).  One  set  of  pilots  may  be  used  randomly  for  a 
variety  of  tests. 

5.2  DATA  COLLECTION  PROCEDURES 

Preparation  for  the  data  collection  phase  of  the  simulator  evaluation 
will  involve  subject  pilot  selection,  development  of  pilot  briefing 
materials,  and  definition  of  the  sequence  of  tests.  This  section 
discusses  these  areas,  as  well  as  the  data  collection  methods  and 
parameter  lists . 
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Prototype  2:  Even  terrain  with  lake  shore  and  river,  major  city  with  one 
ground-level  urban  heliport  and  one  airport  heliport 


5.2.1  Subject  Pilot  Selection 

Pilots  selected  for  this  study  will  include  professional  research 
pilots  selected  from  the  FAA  and  NASA,  and  professional  commercial  and 
private  helicopter  pilots,  with  emphasis  on  those  who  currently 
operate  in  an  urban  environment.  Current  research  pilots,  besides 
serving  as  an  additional  data  source,  will  help  interpret  the 
performance  of  other  subject  pilots,  assist  with  real-time  evaluation 
of  test  parameters,  make  recommendations  for  program  updates,  and 
assist  with  any  other  required/ desired  program  changes  as  data 
processing/analysis  takes  place.  To  maintain  an  appropriate  level  of 
integrity  among  pilot  candidates,  each  will  be  separated  into  well- 
defined  groups  and  categories.  Group  classifications  will  be 
associated  with  pilotage  background  (i.e.,  private,  corporate,  or 
research) ,  while  category  divisions  relate  directly  to  proficiency 
(i.e.,  total  time  as  pilot-in-command). 

5.2.2  Pre-test  Briefings 

Before  conducting  any  tests,  participating  pilots  will  be  thoroughly 
briefed  on  the  objectives  of  the  test.  The  specific  issues  and 
concepts  being  evaluated  will  be  discussed  in  detail.  Charts  and 
other  materials  developed  for  the  tests  will  be  presented  for  review. 

5.2.3  Sequencing  and  Performance  of  Test  Scenarios 

A  plan  will  be  developed  for  the  random  sequence  of  simulator 
scenarios  to  be  flown  by  each  pilot.  This  plan  will  control  the 
effect  of  learning  on  test  results.  This  will  prevent,  for  example, 
the  pilot  from  knowing  where  to  expect  to  find  a  specific  obstacle 
based  simply  on  a  recently  completed  prior  run  (except  those  runs 
whose  purpose  is  to  analyze  the  value  of  familiarity;  in  those  cases, 
a  specific  run  may  be  initiated  to  allow  the  pilot  to  become  very 
familiar  with  the  flight  simulator  capabilities) .  Since  pilot 
availability  is  always  a  limiting  factor,  entire  sequences  of 
scenarios  must  be  presented  to  a  given  pilot  in  a  short  time. 

5.2.4  Logs  and  Parameter  Lists 

The  data  collection  methods  and  logbook  requirements  will  be  developed 
before  the  tests  and  will  be  verified  during  the  simulator  shakedown 
tests.  The  recorded  parameters  will  be  stored  at  an  appropriate 
sample  rate  (twice  per  second)  and  transferred  to  transportable  media, 
such  as  high-density  diskettes  or  magnetic  tapes,  as  required  for 
post-test  processing.  Four  sets. of  parameters  are  of  interest: 

o  Simulator  Operator  Log 

o  Run  sequence  number 
o  Scenario  number 
o  Flight  regime  being  flown 
o  Visibility,  obstructions,  etc. 
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o  Helipad,  obstruction  and  landmark  lighting 

o  Record  of  off-nominal  conditions  experienced  during  the  run 
o  Test  Observer  Log 


o  Run  sequence  number 
o  Subject  pilot  number,  date  and  time 
o  Procedure  under  test 

o  Events  and  commentary  -  turn  points,  deviations,  pilot 
comments,  areas  of  difficulty,  etc. 

o  Pilot  Log 

o  Helicopter  certification/rating 
o  Flight  experience  by  type/hours 
o  Private 
o  Commercial 
o  Airline  transport 
o  Simulator  currency 
o  Simulator  by  type/hour 
o  Debriefing 

o  Recorded  Parameter  List 

o  Run  sequence  number 
o  Time  mark 

o  Environmental  conditions  (winds,  temperatures,  etc.) 
o  Aircraft  position  (X,Y,Z) 
o  Aircraft  velocity  (X,Y,Z) 
o  Sensor  signals  {  altimeters,  etc.) 
o  Course  deviations  (lateral,  vertical) 
o  Control  and  switch  positions 

o  Control  positions  (throttle/engine  speedlevers,  cyclic, 
collective,  etc.) 

5.3  DATA  REDUCTION  AND  ANALYSIS 

The  intent  of  the  planned  task  is  to  collect,  recover,  and  reduce 
simulator  data  to  a  form  useful  to  analysts.  Actual  use  of  this  data 
in  the  development  or  analysis  of  psychological  effects,  airspace 
requirements,  or  target  levels  of  safety  will  be  accomplished  in  phase 
two  of  this  project. 

Data  reduction  will  center  on  assessing  the  influence  of  variables  on 
pilot  performance.  By  establishing  an  analytical  framework  for  the 
evaluation  of  individual  variables,  each  item  can  be  filtered  through 
a  statistical  discrimination  process.  Results  will  offer  a 
foundational  database  for  objective  decisionmaking. 


An  important  factor  in  this  analysis  is  the  variation  in  performance 
of  subject  pilots.  Evaluating  this  element  is  a  paramount  issue. 
Close  investigation  of  "task  variance"  will  further  perfect  the 
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decision  tool.  The  effects  of  variance  will  be  assessed  initially  by 
matching  "between-sub ject"  variables.  Using  flight  experience  as  a 
variable  and  dividing  subject  pilots  (Sj^)  into  groups  on  the  basis  of 
their  flight  time  (figure  2),  provides  the  between-sub ject  category. 
The  variations  in  the  different  simulation  runs  furnishes  the  second 
assessment  arena,  "across-sub ject"  variables.  As  shown  in  figure  2, 
variables  A,  B, . . .N  present  different  scenarios  by  varying  visibility, 
terrain,  or  the  lighting  options.  They  would  be  across-sxab ject 
variables  because  each  would  be  individually  administered  to  subject 
pilots.  Variance  analysis  allows  us  to  test  the  "null  hypothesis"  for 
each  of  the  variables.  In  this  hypothetical  design,  we  would  be  able 
to  substantiate  or  challenge  the  effect  of  individual  variables .  Does 
flight  experience  make  a  difference  when  matched  against  different 
terrain  features?  The  interaction  between  variables  can  be  tested  and 
measured.  By  evaluating  mean  performance,  and  the  differences  between 
the  various  interactions,  the  impact  of  each  treatment  can  be 
demonstrated  and  an  associated  confidence  level  provided.  Each  of  the 
pertinent  measures  could  be  subjected  to  the  "analysis  of  variance," 
providing  usable  measures  of  mental  workload  and  risk. 

Pictorial  flight -presentation  summaries  will  be  plotted  and  assessed. 
These  will  show  plan  and  profile  views  overlaying  a  plot  of  the  flight 
regime  involved,  as  well  as  its  relationship  to  the  obstacles 
underlying  the  VFR  imaginary  surface  for  each  scenario.  A  plot  of 
track  deviation  from  the  ideal  course  will  be  developed  in  the  same 
format  (plan  and  profile)  to  highlight  its  relationship  to  the  actual 
course  flown.  All  presentations  will  be  annotated  with  time  marks  to 
allow  correlation  of  flight  control  data  with  the  data  contained  in 
the  operator  and  observer  logs .  ■ 

Other  data  recorded  as  a  part  of  the  simulator  tests  (including 
primary  flight  control  inputs,  navigation  control  inputs,  and 
instrximent  flags  and  warnings)  will  be  plotted  versus  time  for 
correlation  to  the  graphical  data.  Statistical  analysis  of  pilot 
performance  factors  with  regard  to  the  plan  and  profile  views,  as  well 
as  identification  of  maximum  deviation  or  deflection  events,  also  will 
be  performed.  The  results  will  be  presented  in  tabular  and  graphical 
form. 

6.0  TEST  PLAN  SYLLABUS 

The  test  plan  is  partitioned  into  two  main  categories  that  addresses 
the  technical  support  and  simulation  perspectives  of  the  overall  > 
syllabus.  All  subtask  definitions  and  resource  estimates  in  the  plan 
are  developed  using  a  "bottom-up"  approach. 

The  test  plan  syllabus  is  divided  into  15  distinct  subtasks.  These  15 
subtasks  are  described  briefly  in  the  following  sections.  Appendix  A 
contains  a  generic  timeline  of  the  test  plan  syllabus  to  project  task 
completion  dates. 
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SIMULATOR  VARIABLES 
(e.g.  Visibility,  Terrain,  Lighting  Option.etc) 
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FIGURE  2  SUBSET  VARIABLE  ASSESSMENT 


6.1 


TASK  START -DP 


o  Technical  Support  Contractor  -  This  is  the  initial  task 

development  for  phase  two  of  the  simulation  program.  A  statement 
of  work  (SOW)  and  acquisition  plan  for  simulation  support  services 
will  be  assembled  for  FAA  review  and  approval.  The  level  of 
detail  will  be  of  appropriate  depth  to  complete  the  required 
simulation  task  objectives  as  defined  in  phase  one. 

Deliverable  -  Statement  of  Work  (SOW)  for  simulation  support. 

Travel  -  None. 

6.2  REQUEST  FOR  PROPOSAL  (RFP)  DEVELOPMENT  FOR  SIMULATOR  SUPPORT 

o  Technical  Support  Contractor  -  The  SOW  will  be  reviewed  by  the 
FAA.  The  results  of  the  review  will  be  used  by  the  support 
contractor  to  develop  a  request  for  proposal  (RFP)  to  acquire 
simulation  support  for  the  obstacle-rich  environment  study. 

Deliverable  -  Request  For  Proposal  (RFP) 

Travel  -  None. 

6.3  SIMULATOR  PROPOSAL  EVALUATION 

o  Technical  Support  Contractor  -  The  technical  support  contractor 
will  solicit  proposals  from  qualified  simulation  services 
organizations.  Initial  evaluation  will  be  accomplished  by  the 
technical  support  contractor  to  determine  each  proposal's 
potential.  Results  in  the  form  of  a  priority  list  of  competitors 
will  be  submitted  to  the  FAA  with  a  recommendation  for  final 
selection  and  contract  award. 

Deliverable  -  Technical  evaluation  report  of  proposal  submittal. 

Travel  -  Travel  to  simulation  facilities  will  be  required  to 
assess  bid  proposals.  Team  composition  should  be  limited  to 
primary  engineer/analyst.  Three  working  days  will  be  necessary  at 
each  facility. 

6.4  SIMULATOR  CONTRACT  AWARD 

o  Technical  Support  Contractor  -  Negotiate  a  subcontract  with  the 

selected  simulator  support  supplier.  Areas  of  negotiation  include 
statement  of  work,  scope  of  work,  resource  requirements, 
schedules,  and  cost. 

Deliverable  -  Negotiated  subcontract  for  simulation  services . 
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Travel  -  As  required  to  prospective  simulation  subcontractor's 
facility  for  negotiation. 


6.5  SIMULATION  SCENARIO  SPECIFICATIONS 

o  Technical  Support  Contractor  -  Review  of  simulation  technology 

will  be  necessary  at  this  point  to  ensure  that  scenario  strategy  . 
parallels  simulation  contractor  capabilities.  The  technical 
support  contractor  must  expand  on  the  basic  scenario  provi  e  i 
the  draft  test  plan  and  the  pilot  briefing  material.  A  syllabus 
will  be  developed  to  support  each  scenario  in  the  sunulation  tesr 
plan.  Each  syllabus  will  include  descriptive  and  visual 
specifications  needed  to  assure  that  the  scenario  supports  program 
objectives. 

Deliverable  -  Simulation  scenario  specifications. 


Travel  -  As  required,  for  coordination  of  scenario  specifications 
with  the  simulation  subcontractor. 


o  Simulation  Subcontractor  -  The  technical  support  contrac  or  wi 
provide  the  simulation  subcontractor  with  a  finalized  scenario 
specification  for  scenario  development.  Direct  contact  between 
the  contractors  and  the  FAA  program  manager  is  essential  to 
maintain  task  focus . 


DeliveraKi -  Preliminary  and  final  scenario  programs. 


Travel  —  None . 


6.6  PILOT  QUESTIONNAIRES 

o  Technical  Support  Contractor  -  After  review  of  the  scenario 

programs,  the  support  contractor  will  develop  a  questionnaire  o 
evaluate  pilot  performance  and  visual  recognition  cues.  Direc 
coordination  with  the  simulation  facility  is  necessary.  The  ^ 
central  theme  will  focus  on  assessing  workload  versus  overload  in 
the  execution  of  normal  operational  flight  requirements.  Eac 
task  assessment  will  concentrate  on  human  factors  issues  wit  e 
overall  objective  being  an  evaluation  of  airspace  requirements  in 
an  obstacle-rich  environment. 


Deliverable  -  Pilot  questionnaire. 

Travel  -  As  required  to  the  simulation  subcontractor's  facility. 

o  Simulation  Subcontractor  -  Direct  coordination  between  the  support 
and  simulation  contractors  is  essential.  Development  of  the  pi  o 
questionnaire  must  be  done  in  concert  with  the  simulation 
facility.  This  will  ensure  all  levels  of  the  simulation  pro^ec 
are  assessed  and  evaluated. 
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6.7 


Deliverable  -  Review  and  approval  of  pilot  questionnaire. 

( 

Travel  -  As  required  to  the  technical  support  contractor's 
facility. 

VISUAL  SCENE  DATABASES 

o  Technical  Support  Contractor  -  Based  on  scenario  development 
strategy,  the  associated  visual  database  will  be  designed  to 
programmatically  vary  obstacle  density.  The  technical  support 
contractor  will  guide  and  assist  simulation  engineers  in  the 
development  of  a  visual  scene  database. 

Deliverable  -  None . 

Travel  -  Travel  to  the  simulation  facility  will  be  required.  Team 
composition  should  be  limited  to  primary  engineers/analysts.  No 
more  that  five  working  days  should  be  necessary. 

o  Simulation  Subcontractor  -  The  simulation  contractor  will  be  the 
principal  force  behind  developing  the  visual  scenes  for  the  VFR 
obstacle-rich  scene  environment.  Correlation  between  the  task 
initiative  and  scene  development  programs  must  be  maintained. 
Various  review  stages  are  required  to  keep  task  progression 
focused  to  the  program  objectives. 

Deliverable  -  Visual  scene  database. 

Travel  -  None . 

6.8  FAA  PRE-TEST  REVIEW  AND  EVALUATION 

o  Technical  Support  Contractor  -  Initial  simulation  project 
development  review  will  occur.  Preliminary  results  will  be 
demonstrated  to  the  FAA  program  manager.  A  status  report 
outlining  progress  and  accomplishments  to  date  will  be  presented. 
Both  the  technical  support  contractor  and  the  simulation 
subcontractor  will  be  given  sufficient  lead  time  to  assess  FAA 
comments  before  the  review  meeting.  All  concerned  parties  will 
review  FAA  evaluation  of  task  progress,  direction,  and  status. 

Deliverable  -  Program  review. 

Travel  -  Travel  to  the  simulation  subcontractor's  facility  to 
support  the  review  meeting. 

o  Simulation  Subcontractor  -  The  simulation  subcontractor  will 

support  the  program  review  and  prepare  simulator  demonstrations  as 
required  to  support  the  review. 

Deliverable  -  Simulator  demonstrations  to  support  the  program 
review. 
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Travel  -  None . 


6.9  SUBJECT  PILOT  SCHEDULING 

o  Technical  Support  Contractor  -  In  concert  with  the  scenario 
specifications,  the  pilot  scheduling  format  will  be  expanded 
beyond  the  initial  framework  provided  in  the  pilot  briefing 
material.  This  task  will  be  performed  primarily  by  the  simulation 
siibcontractor  with  support  and  direction  from  the  technical 
support  contractor.  When  initiating  amendments  or  modifications, 
the  technical  support  contractor  must  be  attuned  to  the  simulation 
subcontractor's  facility  schedule. 

Deliverable  -  None. 

Travel  -  Once  the  simulation  portion  of  the  test  plan  begins,  a 
representative  of  the  technical  support  contractor  will  be  in- 
place  at  the  simulation  facility.  This  individual  will  remain  at 
the  simulation  facility  throughout  data  collection  and  data 
analysis  tasks. 

o  Simulation  Subcontractor  -  Based  on  the  pilot  briefing  material, 
the  simulation  subcontractor  will  develop  a  pilot  and  simulator 
schedule  to  execute  the  simulation  tasks.  Subject  pilots  will  be 
provided  by  the  simulation  facility.  Both  the  FAA  and  the 
technical  support  contractor  will  review,  nominate,  and  approve 
pilot  candidates. 

Deliverable  -  Visual  simulation  and  pilot  schedule. 

Travel  -  None . 

6.10  SUBJECT  PILOT  SIMULATION  EVALUATION  (FIRST  TEST  PHASE) 

o  Technical  Support  Contractor  -  The  simulation  event  will  be 

divided  into  two  separate  test  phases .  The  first  test  phase  will 
establish  the  initial  direction  and  course  of  action.  The  second 
test  phase  will  consist  of  adjustments  and  refinements  based  on 
preliminary  data  assessment  results  from  the  first  phase.  The 
technical  support  contractor  will  actively  participate  as  a 
technical  observer  and  provide  a  human  factors  engineer  during 
simulation  runs  and  data  assessment.  The  technical  support 
contractor  will  have  overall  responsibility  for  task  direction  and 
alignment  as  necessary. 

Deliverable  -  None. 

Travel  -  As  stated  previously,  technical  support  personnel  will  be 
in-place  at  the  simulation  facility  throughout  the  test  phase  of 
the  program. 
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o  Simulation  Subcontractor  -  During  the  first  test  phase  of 

simulation,  the  simulation  subcontractdr  will  actively  execute  the 
test  plan.  The  subcontractor  will  keep  all  parties  appraised  of 
task  performance  and  offer  constructive  evaluations. 

Deliverable  -  Test  logs  and  data  from  the  first  test  phase. 

Travel  -  None. 

6.11  DATABASE  CHANGES 

o  Technical  Support  Contractor  -  Preliminary  results  of  the  first 
test  phase  of  simulation  will  be  briefed  to  the  FAA  program 
manager.  The  briefing  will  include  a  status  report  of  initial 
simulation  efforts  and  preliminary  conclusions.  An  evaluation  of 
visual  scene  and  data  collection  revisions  needed  to  fine  tune  or 
readjust  simulation  project  direction  will  be  highlighted. 

Deliverable  -  Briefing  of  first  pilot  simulation  evaluation. 

Travel  -  The  briefing  should  be  scheduled  at  the  simulation 
facility.  This  will  provide  an  opportunity  for  all  parties  to 
visually  assess  the  simulation  and  recommended  changes.  The 
technical  support  contractor  will  require  key  personnel  to  attend. 


o  Simulation  Subcontractor  -  Preliminary  results  from  the  first  test 
phase  will  be  weighed  against  requirements.  Appropriate 
modifications  to  the  scenario  and  questionnaires  will  be  included 
in  the  second  test  phase  programming.  Active  tracking  of  scenario 
and  test  plan  adjustments  will  be  maintained.  Changes  to  visual 
scenes  and  data  collection  parameters  will  be  documented.  A  brief 
description  of  necessary  changes,  if  any,  will  be  provided  to  the 
technical  support  contractor  along  with  an  assessment  of  resource 
and  schedule  impacts. 

Deliverable  -  Description  of  proposed  changes  to  the  tests  with  an 
assessment  of  schedule  and  resource  impacts. 

Travel  -  None. 

6.12  SCENARIO  REFINEMENTS 

o  Technical  Support  Contractor  -  Proposed  changes  to  the  test 
scenarios  provided  by  the  simulation  subcontractor  will  be 
reviewed  by  the  support  contractor.  The  technical  support 
contractor  will  review  the  proposed  changes  and  make  necessary 
recommendations  to  the  FAA  program  manager.  Final  refinements 
will  be  approved  and  reprogramming  accomplished. 

Deliverable  -  Recommended  changes  to  the  test  scenarios . 
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Travel  -  Once  the  proposed  changes  to  the  scenarios  have  been 
approved,  a  representative  of  the  technical  support  contractor 
will  be  in-place  at  the  simulation  facility.  This  individual  will 
remain  at  the  simulation  facility  throughout  the  second  test  phase 
simulation  and  data  analysis. 

o  Simulation  Subcontractor  -  Each  scenario  scene  will  be  modified  as 
approved  to  update  the  VFR  obstacle-rich  environment  program 
requirements.  Preliminary  testing  must  validate  changes  and  match 
test  runs  with  task  objectives. 

Deliverable  -  Modified  test  .scenarios. 

Travel  -  None. 

6.13  SUBJECT  PILOT  SIMULATION  EVALUATION  (SECOND  TEST  PHASE) 

o  Technical  Support  Contractor  -  The  support  contractor  will 

actively  participate  as  a  technical  observer  and  provide  a  human 
factors  engineer  during  the  second  half  of  the  simulation  runs  and 
data  assessment.  The  technical  support  contractor  will  have 
overall  responsibility  for  task  direction  and  alignment  as 
necessary. 

Deliverable  -  None. 

Travel  -  As  stated  previously,  technical  support  personnel  will  be 
in-place  at  the  simulation  facility  throughout  the  test  phase  of 
the  program. 

o  Simulation  Subcontractor  -  In  coordination  with  the  test  plan 
timeline  schedule,  a  precise  simulation  program  will  be 
maintained.  The  simulation  siabcontractor  will  keep  all  concerned 
parties  appraised  of  task  performance  and  offer  constructive 
evaluations . 

Deliverable  -  Test  logs  and  data  for  the  second  test  phase. 

Travel  -  None 

6.14  DATA  ANALYSIS  AND  PRESENTATION 

o  Technical  Support  Contractor  -  Based  on  the  initial  results,  the 
support  contractor  will  brief  the  FAA  program  manager.  The 
briefing  should  address  all  facets  of  the  simulation  program. 

Final  results  compiled  after  assessment  and  reduction  will  be 
incorporated  into  a  draft  technical  letter  report  and  a  final 
FAA/industry  presentation. 

Deliverables  -  Initial  and  final  briefing  presentations  on  the 
results  of  the  entire  test  program;  draft  technical  report  on  the 
results  of  the  testing. 
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Travel  —  The  preliminary  briefing  will  be  conducted  at  the 
simulation  facility.  The  final  briefing  should  be  scheduled  at  a 
central  location.  Travel  will  be  contingent  on  the  briefing 
location. 

o  Simulation  Subcontractor  -  Assistance  will  be  provided  to  the 

technical  support  contractor  to  detail  simulation  results .  Data 
assessment  and  reduction  will  be  the  responsibility  of  the 
simulation  subcontractor.  Results  will  be  provided  to  the  support 
contractor  in  a  preliminary  simulation  report.  Review  of  briefing 
material  will  be  required  by  the  simulation  contractor  to  ensure 
data  results  are  appropriately  addressed. 

Deliverable  -  Preliminary  simulation  report. 

Travel  —  Travel  to  the  technical  support  contractor  as  required  to 
present  the  preliminary  simulation  report. 

6.15  FINAL  REPORT 

o  Technical  Support  Contractor  -  All  information  and  data  gathered 
during  this  project  will  be  assembled  and  documented  in  a  final 
technical  report. 

Deliverable  -  Final  technical  report. 

Travel  -  None 

o  Simulation  Subcontractor  —  The  simulation  subcontractor  will 

provide  an  editorial  review  of  the  final  technical  report  before 
delivery  to  the  FAA  program  manager.  Findings  will  be  presented 
to  the  technical  support  contractor. 

Deliverable  -  Preliminary  review  of  final  report. 

Travel  -  Travel  to  the  technical  support  contractor  as  required  to 
present  the  findings. 
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APPENDIX  A 

DRAFT  TEST  PLAN  SCHEDULE 
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